[SPARK-31744][R][INFRA] Remove Hive dependency in AppVeyor build temporarily#28564
Closed
HyukjinKwon wants to merge 1 commit intoapache:masterfrom
Closed
[SPARK-31744][R][INFRA] Remove Hive dependency in AppVeyor build temporarily#28564HyukjinKwon wants to merge 1 commit intoapache:masterfrom
HyukjinKwon wants to merge 1 commit intoapache:masterfrom
Conversation
|
Test build #122774 has finished for PR 28564 at commit
|
Member
Author
|
AppVeyor tests passed. The skipped tests are same: and given this run it speeded up to 10 ~ 20 mins roughly. |
Member
Author
|
@dongjoon-hyun can you take a quick look please? |
dongjoon-hyun
approved these changes
May 18, 2020
Member
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM. Thank you for the fix, @HyukjinKwon .
Merged to master/3.0.
Member
Author
|
Thank you! |
dongjoon-hyun
pushed a commit
that referenced
this pull request
May 18, 2020
…orarily ### What changes were proposed in this pull request? This PR targets to remove Hive profile in SparkR build at AppVeyor in order to: - Speed up the build. Currently, SparkR build is [reaching the time limit](https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/32853533). - Clean up the build profile. ### Why are the changes needed? We're hitting a time limit issue again and this PR could reduce the build time. Seems like we're [already skipping Hive related tests in SparkR](https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/32853533) for some reasons, see below: ``` test_sparkSQL.R:307: skip: create DataFrame from RDD Reason: Hive is not build with SparkSQL, skipped test_sparkSQL.R:1341: skip: test HiveContext Reason: Hive is not build with SparkSQL, skipped test_sparkSQL.R:2813: skip: read/write ORC files Reason: Hive is not build with SparkSQL, skipped test_sparkSQL.R:2834: skip: read/write ORC files - compression option Reason: Hive is not build with SparkSQL, skipped test_sparkSQL.R:3727: skip: enableHiveSupport on SparkSession Reason: Hive is not build with SparkSQL, skipped ``` Although we build with Hive profile. So, the Hive profile is useless here. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? AppVeyor will test it out. Closes #28564 from HyukjinKwon/SPARK-31744. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit f352cef) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR targets to remove Hive profile in SparkR build at AppVeyor in order to:
Why are the changes needed?
We're hitting a time limit issue again and this PR could reduce the build time.
Seems like we're already skipping Hive related tests in SparkR for some reasons, see below:
Although we build with Hive profile. So, the Hive profile is useless here.
Does this PR introduce any user-facing change?
No, dev-only.
How was this patch tested?
AppVeyor will test it out.